Experiments in Interpretation-Guided Segmentation

This paper presents a new approach for integrating the segmentation and interpretation phases of scene analysis. Knowledge from a variety of sources is used to make inferences about the interpretations of regions, and regions are merged in accordance with their possible interpretations. The deduction of region interpretations is performed using a generalization of Waltz's filtering algorithm. Deduction proceeds by eliminating possible region interpretations that are not consistent with any possible interpretation of an adjacent region. Different sources of knowledge are expressed uniformly as constraints on the possible interpretations of regions. Multiple sources of knowledge can thus be combined in a straightforward way such that incremental additions of knowledge (or equivalently, human guidance) will effect incremental improvements in performance. Experimental results are reported in three scene domains, landscapes, mechanical equipment, and rooms, using, respectively, a human collaborator, a geometric model and a set of relational constraints as sources of knowledge. These experiments demonstrate that segmentation is much improved when integrated with interpretation. Moreover, the integrated approach incurs only a small computational overhead over unguided segmentation. Applications of the approach in cartography, photointerpretation, vehicle guidance, medicine, and motion picture analysis are suggested.